Guidance acquisition device, guidance acquisition method, and program

ABSTRACT

Face image data is acquired and a face image is captured, and a difference between a face image indicated by the face image data and the face image that is captured or a candidate of the difference is detected on the basis of at least one of the face image indicated by the face image data that is acquired, and the face image that is captured. Guidance is acquired on the basis of the difference or the candidate of the difference which is detected, and an output unit is controlled to output the guidance.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 16/884,801 filed on May 27, 2020, which is acontinuation application of U.S. patent application Ser. No. 15/758,136filed on Mar. 7, 2018, which is issued as U.S. Pat. No. 10,706,266,which is a National Stage Entry of international applicationPCT/JP2016/074639 filed on Aug. 24, 2016, which claims the benefit ofpriority from Japanese Patent Application 2015-177813 filed on Sep. 9,2015, the disclosures of all of which are incorporated in their entiretyby reference herein.

TECHNICAL FIELD

The present invention relates to a guidance acquisition device, aguidance acquisition method, and a program

BACKGROUND ART

As one of identity verification methods such as for management ofentrance into and exit from a predetermined area, face authentication isknown. In face authentication, a face image of an authentication targetis captured in advance, and the face image is compared with a face imagecaptured in advance to perform determination of identity.

Here, several technologies for improving accuracy of the faceauthentication are suggested. For example, a face collation devicedescribed in Patent Document 1 includes a photographing unit thatcaptures a face image of a person to be photographed, a light directiondetection unit that detects a direction of the brightest light in frontof the photographing unit, and a guide unit that guides the person to bephotographed so as to stand in the direction of the light on the basisof the light direction that is detected. Patent Document 1 describesthat even in a case where the face collation device is provided outdoorsor in a location in which external light shines, the person to bephotographed is guided to an appropriate photographing position, andthus an appropriate face image can be captured according to theabove-described configuration.

CITATION LIST Patent Literature

[Patent Document 1]

-   Japanese Unexamined Patent Application, First Publication No.    2008-4003

SUMMARY OF INVENTION Technical Problem

As a cause of a problem in which the face authentication cannot beappropriately performed, various causes are considered in addition tolight direction, and thus a way to handle failure of the faceauthentication are different depending on the causes of a failure of theface authentication. However, there is a possibility that it isdifficult to understand what is a cause when observing from an outerside of a device that performs the face authentication, and thus anappropriate response may not be performed. Particularly, there is apossibility that the authentication target cannot understand the way tohandle the failure and feel stress, and some time may be taken untilauthentication is accepted.

An object of the invention is to provide a guidance acquisition device,a guidance acquisition method, and a program which are capable ofsolving the above-described problems.

Solution to Problem

According an aspect of the invention, there is provided a guidanceacquisition device including: a data acquisition unit that is configuredto acquire face image data; a photographing unit that is configured tocapture a face image; a difference detection unit that is configured todetect a difference between a face image indicated by the face imagedata acquired by the data acquisition unit and the face image capturedby the photographing unit or a candidate of the difference on the basisof at least one of the face image indicated by the face image dataacquired by the data acquisition unit and the face image captured by thephotographing unit; a guidance acquisition unit that is configured toacquire guidance on the basis of the difference or the candidate of thedifference which is detected by the difference detection unit; and anoutput control unit that is configured to control an output unit tooutput the guidance that is acquired by the guidance acquisition unit.

According to another aspect of the invention, there is provided aguidance acquisition method that is executed by a guidance acquisitiondevice including a correlation information storage unit that isconfigured to store correlation information that indicates a correlationbetween a difference between a face image indicated by face image dataand a face image obtained through photographing, and guidance for aperson to be photographed with respect to the difference, in advance.The method includes: a data acquisition step of acquiring the face imagedata; a photographing step of capturing the face image; a differencedetection step of detecting the difference between the face imageindicated by the face image data acquired in the data acquisition step,and the face image captured in the photographing step; a guidanceacquisition step of acquiring guidance that represents guidancecorrelated with the difference detected in the difference detection stepin the correlation information; and an output control step ofcontrolling an output unit to output the guidance acquired in theguidance acquisition step.

According to still another aspect of the invention, there is provided aprogram that causes a computer including a photographing unit, and acorrelation information storage unit that is configured to storecorrelation information that indicates a correlation between adifference between a face image indicated by face image data and a faceimage obtained through photographing, and guidance for a person to bephotographed with respect to the difference, in advance to execute: adata acquisition step of acquiring the face image data; a photographingstep of capturing the face image by the photographing unit; a differencedetection step of detecting the difference between the face imageindicated by the face image data acquired in the data acquisition step,and the face image captured in the photographing step; a guidanceacquisition step of acquiring guidance correlated with the differencedetected in the difference detection step in the correlationinformation; and an output control step of controlling an output unit tooutput the guidance acquired in the guidance acquisition step.

Advantageous Effects of Invention

According to the invention, in a case where a device that is configuredto perform face authentication cannot appropriately face authentication,it is possible to increase the probability in which an authenticationtarget can understand a way to handle a failure of the faceauthentication.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic block diagram showing a functional configurationof a face authentication device according to an embodiment of theinvention.

FIG. 2 is a view showing an example of a display screen image of adisplay unit in a state in which the face authentication device of theembodiment is in execution of the face authentication.

FIG. 3 is a view showing an example of a display screen image of thedisplay unit in a case where an error occurs in the face authenticationof the face authentication device of the embodiment.

FIG. 4 is a view showing an example of correlation information that isstored in a correlation information storage unit of the embodiment.

FIG. 5 is a view showing an example of an image that is captured by aphotographing unit in a case where a distance between a person to bephotographed and the photographing unit is long in the embodiment.

FIG. 6 is a view showing an example of an image that is captured by thephotographing unit in a case where a position of the person to bephotographed deviates in a horizontal direction in the embodiment.

FIG. 7 is a flowchart showing an example of a processing procedure thatis performed by the face authentication device of the embodiment.

FIG. 8 is a schematic block diagram showing a minimum configuration of aguidance acquisition device according to the invention.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment of the invention will be described, but thefollowing embodiment does not limit the invention in the followingappended claims. In addition, it cannot be said that the entirety ofcombinations of characteristics described in the embodiment areessential for the solution to problem.

FIG. 1 is a schematic block diagram showing a functional configurationof a face authentication device in an embodiment of the invention. Inthe drawing, a face authentication device 100 includes a dataacquisition unit 110, a photographing unit 120, an output unit 130, astorage unit 170, and a control unit 180.

The output unit 130 includes a display unit 131 and a voice output unit132.

The storage unit 170 includes a correlation information storage unit171.

The control unit 180 includes a difference detection unit 181, aguidance acquisition unit 182, a face authentication unit 183, and anoutput control unit 184.

The face authentication device 100 is a device that is configured toperform identity verification as face authentication. Hereinafter, adescription will be provided of a case where the face authenticationdevice 100 performs identity verification during immigration examinationat an airport or the like as an example.

However, application fields of the face authentication device 100 arenot limited to the immigration examination. For example, the faceauthentication device 100 may be a device that verifies whether or notauthority of entrance into and exit from a specific facility is presentthrough face authentication. In this case, the face authenticationdevice 100 may read out face image data from identification paper (forexample, an ID card) that is carried by an authentication target, andmay compare the face image data with a captured image to perform theface authentication. Alternatively, the face authentication device 100may store face image data of an entrance/exit-authorized person inadvance.

In addition, the face authentication device 100 corresponds to anexample of a guidance acquisition device, and output guidance in a casewhere an error occurs in the face authentication. The faceauthentication error stated here represents a case where a faceauthentication result cannot be obtained, that is, a case where the faceauthentication device 100 cannot determine whether or not anauthentication target is the same person as a person shown in faceauthentication data. In addition, the guidance that is output by theface authentication device 100 is information that shows a method ofhandling the face authentication error, by the authentication target.

For example, the face authentication device 100 is realized when acomputer on which a camera and the like are mounted executes a program.Alternatively, the face authentication device 100 may be configured byusing dedicated hardware.

The data acquisition unit 110 acquires face image data. Specifically,the data acquisition unit 110 includes a passport reader device, andreads out face image data, which is registered in an IC (integratedcircuit) chip in advance, from the IC chip that is embedded in thepassport.

In addition, the data acquisition unit 110 acquires attribute dataindicating attributes such as nationality, sexuality, and an age of anauthentication target (a person to be captured by the photographing unit120 (hereinafter, simply referred to a person to be photographed)) fromthe IC chip embedded in the passport in addition to the face image data.The attributes of the authentication target stated here representsinnate characteristics of the authentication target (that is,characteristics which are determined when the authentication target wasborn). Examples of the attributes of the authentication target includenationality, sexuality, and age as described above.

Furthermore, the data acquisition unit 110 may acquire at least any onebetween the face image data and the attribute data from elsewhere otherthan from a passport.

For example, in a case where the face authentication device 100 is adevice that verifies whether or not authority of entrance into and exitfrom a specific facility is present through face authentication, thedata acquisition unit 110 may read out face image data (face image dataindicating a face image of an authentication target) of theauthentication target and attribute data (attribute data indicating anattribute of the authentication target) of the authentication target,from identification paper (for example, an ID card).

Alternatively, the storage unit 170 may store the face image data andthe attribute data in advance (before the face authentication device 100performs the face authentication), and the data acquisition unit 110 mayread out the face image data of the authentication target and theattribute data of the authentication target from the storage unit 170.

Furthermore, acquisition of the attribute data of the authenticationtarget by the data acquisition unit 110 is not an essential requirement.The data acquisition unit 110 may acquire at least the face image data.

The photographing unit 120 is provided with a camera to performphotographing. Particularly, the photographing unit 120 repetitivelycaptures the face image of the authentication target. The repetitivephotographing stated here may be photographing capable of repetitivelyobtaining the face image. For example, the photographing unit 120 mayphotograph a moving image, and may repeat capturing of a still image atpredetermined time intervals (for example, every second).

The output unit 130 outputs information. Particularly, the output unit130 outputs guidance.

The output of the guidance stated here may be an output of a signalindicating the guidance. Alternatively, the output of the guidancestated here may be presentation of guidance in an aspect in which theauthentication target can understand guidance such as display ofguidance and voice output of guidance.

For example, the display unit 131 includes a display screen such as aliquid crystal panel and a light-emitting diode (LED) panel, anddisplays various images. Particularly, the display unit 131 displays aface image indicated by face image data that is acquired by the dataacquisition unit 110, and a face image (face image obtained throughphotographing performed by the photographing unit 120) that is capturedby the photographing unit 120. In addition, the display unit 131displays guidance.

Display of the guidance by the display unit 131 corresponds to anexample of the output of the guidance.

FIG. 2 is a view showing an example of a display screen image of thedisplay unit 131 in a state in which the face authentication device 100is in execution of the face authentication. An image P11 represents anexample of a face image (a face image indicated by face image data thatis acquired by the data acquisition unit 110) of a passport. The faceimage of the passport is an image that is used in face authentication.When the display unit 131 displays the face image of the passport, and aperson to be photographed (authentication target) adjusts their facialexpression, their position, and the like to the facial expression,position, and the like at the time of capturing the face image of thepassport, it is possible to reduce the probability of an error occurringin the face authentication.

In addition, the image P12 represents an example of an image obtained byhorizontally inverting an image captured by the photographing unit 120.Since the display unit 131 displays an image that is obtained byhorizontally inverting the latest image captured by the photographingunit 120, in a case where a position of the person to be photographeddeviates in a horizontal direction with respect to a position of thephotographing unit 120 (a photographing range of the photographing unit120), the person to be photographed can intuitively understand to whichof a left direction or a right direction is deviated. Furthermore, theimage P12 may display the latest image captured by the photographingunit 120 as is without the horizontal inversion.

In addition, the display unit 131 displays a message of “your face isbeing photographed” in a region A11. Since the display unit 131 displaysthe message, the person to be photographed can understand that the faceauthentication device 100 is in execution of the face authentication.

FIG. 3 is a view showing an example of a display screen image of thedisplay unit 131 in a case where an error occurs in the faceauthentication by the face authentication device 100. An image P21represents an example of the face image (face image indicated by theface image data that is acquired by the data acquisition unit 110) ofthe passport. In addition, an image P22 represents an example of animage that is used in the face authentication by the face authenticationdevice 100 among images captured by the photographing unit 120. Both ofthe images P21 and P22 are images which are used in the faceauthentication by the face authentication device 100. As describedabove, since the display unit 131 displays the two images which are usedin the face authentication, the authentication target can estimate thecause of an error by comparing the two images.

In addition, the image P23 represents an example of an image obtained byhorizontally inverting the latest image captured by the photographingunit 120. When the display unit 131 displays the face image of thepassport and the latest image captured by the photographing unit 120,and the authentication target detects a difference between the twoimages and takes countermeasures to reduce the difference, it ispossible to reduce the probability of an error occurring in the faceauthentication performed by the face authentication device 100.

Furthermore, the image P23 may be an image which is displayed as it iswithout horizontal inverting the latest image captured by thephotographing unit 120.

In addition, the display unit 131 displays a message of “please, comecloser” in a region A21. The message corresponds to an example of theguidance, and shows a handling method in which the person to bephotographed approaches the camera of the photographing unit 120.

In addition, the display unit 131 displays an arrow B21. The arrow B21also corresponds to an example of the guidance, and shows a handlingmethod in which the person to be photographed approaches the camera ofthe photographing unit 120 as in the message in the region A21.

As described above, when the display unit 131 displays the guidancethrough display of an icon such as an arrow, various persons to bephotographed, who use different languages, can understand the guidance.For example, when the display unit 131 displays a movement direction ofthe person to be photographed with an arrow, various persons to bephotographed, who use different languages, can understand the movementdirection.

The voice output unit 132 includes a speaker and outputs sound.Particularly, the voice output unit 132 outputs the guidance with avoice.

Furthermore, a method of outputting the guidance by the faceauthentication device 100 is not limited to display or voice output ofthe guidance. For example, the output unit 130 may be constituted as adevice independent from the face authentication device 100, and the faceauthentication device 100 may allow the output unit 130 to output(transmit) a signal indicating the guidance.

The storage unit 170 is constituted by using a storage device that isprovided in the face authentication device 100, and stores variouspieces of information.

The correlation information storage unit 171 stores correlationinformation. The correlation information that is stored in thecorrelation information storage unit 171 is information indicating acorrelation between a difference between the face image (face imageindicated by the face image data acquired by the data acquisition unit110) of the passport and the face image captured by the photographingunit 120, and the guidance for the person to be photographed with regardto the difference.

FIG. 4 is a view showing an example of the correlation information thatis stored in the correlation information storage unit 171.

The correlation information storage unit 171 shown in the drawing has atable-type data structure, and a difference column and a handling methodcolumn are included in each row.

The difference column is a column in which information indicating adifference between the face image of the passport and the face imagecaptured by the photographing unit 120 is stored. In addition, thehandling method column is a column in which information indicating ahandling method of the person to be photographed (authentication target)with regard to the difference shown in the difference column is stored.The information stored in the handling method column corresponds to anexample of the guidance for the person to be photographed.

As described above, in the correlation information, a difference betweenthe face image indicated by the face image data acquired by the dataacquisition unit 110 and the face image captured by the photographingunit 120, and the guidance for the person to be photographed with regardto the difference are correlated in every row.

For example, in a case where the face image of the passport and the faceimage captured by the photographing unit 120 are different in facialexpression, as a handling method of the person to be photographed, itcan be considered that the person to be photographed shows the samefacial expression as in the face image of the passport such as beingexpressionless. Accordingly, as a handling method in a case where thedifference is “facial expression”, the correlation information in FIG. 4shows a handling method of “please, be expressionless”.

In addition, as a handling method in a case where it is difficult toimmediately eliminate the difference between the face image of thepassport and the face image captured by the photographing unit 120, thecorrelation information in FIG. 4 shows a handling method of “please,notify an attendant”.

For example, in a case where a facial hair is not present in the faceimage of the passport, but the facial hair is shown in the face imagecaptured by the photographing unit 120, it is considered that it isdifficult to take countermeasures by saving off the facial hair in thatsituation. Accordingly, as a handling method for a case where thedifference is “facial hair”, the correlation information in FIG. 4 showsa handling method of “please, notify an attendant”. When the faceauthentication device 100 outputs (for example, displays) the handlingmethod, the authentication target can be made aware that a handlingmethod is contacting the attendant to inquire a countermeasure method.

Furthermore, the correlation information that is stored in thecorrelation information storage unit 171 is not limited to storage ofinformation indicating a handling method of the person to bephotographed as guidance for the person to be photographed. For example,the correlation information storage unit 171 may store correlationinformation that includes information indicating a difference betweenthe face image of the passport and the face image captured by thephotographing unit 120 as the guidance for the person to bephotographed. In this case, the information indicating the differencebetween the face image of the passport and the face image captured bythe photographing unit 120 corresponds to an example of the guidance.

For example, the display unit 131 may display the face image of thepassport, and guidance of “the facial expression is different”. In thiscase, the person to be photographed can understand and execute ahandling method of showing the same facial expression as in the faceimage of the passport with reference to the face image of the passport.

The control unit 180 controls respective units of the faceauthentication device 100 to perform various kinds of processing. Forexample, the control unit 180 is realized when a central processing unit(CPU) of the face authentication device 100 reads out a program from thestorage unit 170 and executes the program.

The difference detection unit 181 detects the difference between theface image indicated by the face image data acquired by the dataacquisition unit 110, and the face image captured by the photographingunit 120, or a candidate of the difference on the basis of at least oneof the face image indicated by the face image data acquired by the dataacquisition unit 110, and the face image captured by the photographingunit 120.

Particularly, the difference detection unit 181 may detect the candidateof the difference on the basis of the face image indicated by the faceimage data acquired by the data acquisition unit 110. For example, in acase where the face image of the passport which is acquired by the dataacquisition unit 110 is an eyeglass wearing image, the differencedetection unit 181 may detect wearing or non-wearing of eyeglasses asthe candidate of the difference.

In a case where the difference detection unit 181 detects the candidate,for example, the display unit 131 displays the face image of thepassport in a manner of emphasizing eyeglasses, and thus the person tobe photographed can be made aware of necessity to wear eyeglasses. Whenthe person to be photographed, who does not wear eyeglasses, wearseyeglasses, it is possible to reduce the probability of an erroroccurring in the face authentication performed by the faceauthentication device 100.

In addition, in this case, even in a state in which the photographingunit 120 does not photograph the face of the person to be photographed,the difference detection unit 181 can detect the candidate of thedifference. Accordingly, at an early stage, for example, before theperson to be photographed stands within a photographing range of thephotographing unit 120, the display unit 131 can display a suggestion towear eyeglasses.

Alternatively, the difference detection unit 181 may detect thedifference between the face image indicated by the face image dataacquired by the data acquisition unit 110, and the face image capturedby the photographing unit 120, or the candidate of the difference on thebasis of the face image captured by the photographing unit 120. Forexample, in a case where the face image captured by the photographingunit 120 is a mask-wearing face image, the difference detection unit 181may detect wearing or non-wearing of the mask as the candidate of thedifference. In a case where the difference detection unit 181 detectsthe candidate, for example, the display unit 131 displays a message of“please, take off the mask”, and thus the person to be photographed canbe made aware of necessity to take off the mask. When the person to bephotographed takes off the mask, it is possible to reduce theprobability of an error occurring in the face authentication that isperformed by the face authentication device 100.

In addition, in this case, even in a state in which the data acquisitionunit 110 does not acquire data of the face image, the differencedetection unit 181 can detect the candidate of the difference.Accordingly, at an early stage, for example, before the person to bephotographed brings the passport into contact with a passport readerdevice, the display unit 131 can display a suggestion to take off themask.

Furthermore, the difference detection unit 181 may detect the candidateof the difference other than the wearing or non-wearing of the mask bydetecting that the face image captured by the photographing unit 120faces a lateral side.

Alternatively, the difference detection unit 181 may detect thedifference between the face image indicated by the face image dataacquired by the data acquisition unit 110 and the face image captured bythe photographing unit 120. Particularly, in a case where an erroroccurs in the face authentication that is performed by the faceauthentication unit 183, the difference detection unit 181 detects adifference that becomes the cause of the face authentication error, thedifference being the difference between the face image indicated by theface image data acquired by the data acquisition unit 110 and the faceimage captured by the photographing unit 120. A description will beprovided of the difference that is detected by the difference detectionunit 181 with reference to FIG. 5 and FIG. 6 .

FIG. 5 is a view showing an example of an image that is captured by thephotographing unit 120 in a case where a distance between the person tobe photographed and the photographing unit 120 is long. In the drawing,an example of a display screen image (image that is displayed on thedisplay screen by the display unit 131) of the display unit 131 isshown. An image P31 represents an example of a face image (face imageindicated by the face image data acquired by the data acquisition unit110) of the passport. In addition, an image P32 represents an example ofan image obtained by horizontally inverting an image captured by thephotographing unit 120. In an example of FIG. 5 , the photographing unit120 is capturing an image of the person to be photographed whichincludes a face image (image P33) of the person to be photographed.

For example, the difference detection unit 181 detects a position ofeyes in the image captured by the photographing unit 120, and extracts aface portion in the image captured by the photographing unit 120.

In addition, the difference detection unit 181 determines whether or nota gap size (length indicated by an arrow B31) between a left eye and aright eye is smaller than a gap threshold value, and whether or not asize of the face portion is smaller than a face portion threshold value.Here, the gap threshold value and the face portion threshold value arepredetermined threshold values (for example, constants) which are set asthreshold values in determination as to whether or not the distancebetween the person to be photographed and the photographing unit 120 islong. In addition, the size of the face portion may be any one of anarea of the image of the face portion, a vertical length of the image ofthe face portion, and a horizontal width of the image of the faceportion, or a combination thereof.

Between two conditions such things as the size of the gap between theleft eye and the right eye is smaller than the gap threshold value, andthe size of the face portion is smaller than the face portion thresholdvalue, in a case where it is determined that one or more conditions aresatisfied, the difference detection unit 181 determines (detects) thatthe distance between the person to be photographed and the photographingunit 120 is long.

Furthermore, the difference in which the distance between the person tobe photographed and the photographing unit 120 is long corresponds to adifference of “distance (far)” in the example of FIG. 4 .

Furthermore, a method in which the difference detection unit 181 detectsthe difference in which the distance between the person to bephotographed and the photographing unit 120 is long is not limited tothe above-described method. For example, the difference detection unit181 may detect the difference in which the distance between the personto be photographed and the photographing unit 120 is long on the basisof only any one of the gap size between the left eye and the right eye,and the size of the face portion.

FIG. 6 is a view showing an example of an image that is captured by thephotographing unit 120 in a case where a position of the person to bephotographed deviates in a horizontal direction. The drawing shows anexample of a display screen image of the display unit 131. An image P41represents an example of a face image (face image acquired by the dataacquisition unit 110) of the passport. In addition, an image P42represents an example of an image obtained by horizontally inverting animage captured by the photographing unit 120. In the example of FIG. 6 ,the photographing unit 120 is capturing an image of the person to bephotographed which includes a face image (image P43) of the person to bephotographed. In addition, an x-axis is a coordinate axis in ahorizontal direction in the image P42. Ay-axis is a coordinate axis in avertical direction in the image P42.

For example, the difference detection unit 181 detects a position ofeyes in the image captured by the photographing unit 120, and extracts aface portion in the image captured by the photographing unit 120.

In addition, the difference detection unit 181 determines whether or notthe x-coordinate of the left eye is greater than a rightward deviationthreshold value of the left eye coordinates, whether or not thex-coordinate of the right eye is greater than a rightward deviationthreshold value of the right eye coordinates, and whether or not anx-coordinate of the face portion (for example, an average value of anx-coordinate on a left end and a y-coordinate on a right end in a regionextracted as the face portion) is greater than a rightward deviationthreshold value of the face portion coordinates.

Here, the rightward deviation threshold value of the left eyecoordinates, the rightward deviation threshold value of the right eyecoordinates, and the rightward deviation threshold value of the faceportion coordinates are predetermined threshold values (for example,constants) which are set as threshold values in determination as towhether or not a position of the person to be photographed deviates tothe right side with respect to the photographing unit 120 when seen fromthe person to be photographed.

Among three conditions including a condition in which the x-coordinateof the left eye is greater than the rightward deviation threshold valueof the left eye coordinates, a condition in which the x-coordinate ofthe right eye is greater than the rightward deviation threshold value ofthe right eye coordinates, and a condition in which the x-coordinate ofthe face portion is greater than the rightward deviation threshold valueof the face portion coordinates, in a case where it is determined thatone or more conditions are satisfied, the difference detection unit 181determines (detects) that the position of the person to be photographeddeviates to a right side with respect to photographing unit 120 whenseen from the person to be photographed.

Similarly, among three conditions including a condition in which thex-coordinate of the left eye is less than a leftward deviation thresholdvalue of the left eye coordinates, a condition in which the x-coordinateof the right eye is less than a leftward deviation threshold value ofthe right eye coordinates, and a condition in which the x-coordinate ofthe face portion is less than a leftward deviation threshold value ofthe face portion coordinates, in a case where it is determined one ormore conditions are satisfied, the difference detection unit 181determines (detects) that the position of the person to be photographeddeviates to a left side with respect to photographing unit 120 when seenfrom the person to be photographed.

Here, the leftward deviation threshold value of the left eyecoordinates, the leftward deviation threshold value of the right eyecoordinates, and the leftward deviation threshold value of the faceportion coordinates are predetermined threshold values (for example,constants) which are set as threshold values in determination as towhether or not the position of the person to be photographed deviates tothe left side with respect to the photographing unit 120 when seen fromthe person to be photographed.

Furthermore, the difference in which the position of the person to bephotographed deviates to the right side with respect to thephotographing unit 120 when seen from the person to be photographedcorresponds to a positional deviation (right) in a “positional deviation(right and left)” in the example of FIG. 4 .

In this case, the output unit 130 outputs (for example, displays) ahandling method of “please, move to the left side”.

In addition, the difference in which the position of the person to bephotographed deviates to the left side with respect to the photographingunit 120 when seen from the person to be photographed corresponds to apositional deviation (left) in the “positional deviation (right andleft)” in the example of FIG. 4 .

In this case, the output unit 130 outputs (for example, displays) ahandling method of “please, move to the right side”.

Furthermore, a method in which the difference detection unit 181 detectsa difference in which the position of the person to be photographeddeviates in a horizontal direction is not limited to the above-describedmethod.

For example, the difference detection unit 181 may detect the differencein which the position of the person to be photographed deviates in thehorizontal direction (deviates to a left side, or deviates to a rightside) on the basis of one or more among an x-coordinate of a left ear,an x-coordinate of a right ear, an x-coordinate of a nose, and anx-coordinate of a mouth instead of the x-coordinate of the left eye, thex-coordinate of the right eye, and the x-coordinate of the face portionor in addition to one or more among the x-coordinates.

Furthermore, as the x-coordinate of the face portion, the differencedetection unit 181 may use an x-coordinate of the center of the portion,or an x-coordinate of a left end of the portion, or an x-coordinate of aright end of the portion.

The guidance acquisition unit 182 acquires guidance on the basis of thedifference that is detected by the difference detection unit 181, or acandidate of the difference.

For example, as described above, in a case where the differencedetection unit 181 detects that the face image of the passport is aneyeglass-wearing face image, the guidance acquisition unit 182 mayacquire information indicating processing of displaying the face imageof the passport in a manner of emphasizing eyeglasses. In order for thedifference detection unit 181 to acquire the information, the differencedetection unit 181 or the storage unit 170 stores information, in whicha candidate of the difference in which the face image of the passport isthe eyeglass-wearing face image (candidate of the difference betweenwearing and non-wearing of eyeglasses), and the processing of displayingthe face image of the passport in the manner of emphasizing theeyeglasses are correlated with each other, in advance.

Furthermore, displaying of the face image of the passport in the mannerof emphasizing the eyeglasses corresponds to an example of the guidancethat suggests the person be photographed wearing eyeglasses.

Alternatively, as described above, in a case where the differencedetection unit 181 detects that the face image captured by thephotographing unit 120 is a mask-wearing face image, the guidanceacquisition unit 182 may acquire a message of “please, take off themask”. In order for the difference detection unit 181 to acquire themessage, the difference detection unit 181 or the storage unit 170 maystore information, in which a candidate of the difference correspondingto the fact that the face image captured by the photographing unit 120is the mask-wearing face image (candidate of the difference betweenwearing and non-wearing of the mask), and the message of “please, takeoff the mask” are correlated with each other, in advance.

Furthermore, the message “please, take off the mask” corresponds to anexample of guidance that suggests the person be photographed without amask.

Alternatively, as described above, in a case where the differencedetection unit 181 detects that the face image captured by thephotographing unit 120 is a lateral face image, the guidance acquisitionunit 182 may acquire a message of “please, face the front”.Alternatively, in this case, the guidance acquisition unit 182 mayacquire information indicating processing of presenting a position ofthe camera to the person to be photographed.

Alternatively, the guidance acquisition unit 182 acquires guidance thatis correlated with the difference detected by the difference detectionunit 181 in correlation information.

For example, the correlation information storage unit 171 storescorrelation information in which a difference between the image of thepassport and the image that is captured by the photographing unit 120,and a message as guidance are correlated. In addition, the guidanceacquisition unit 182 fetches the message as the guidance from thecorrelation information to acquire the guidance.

Alternatively, the guidance acquisition unit 182 may acquire theguidance by a method other than the method of fetching the guidance fromthe correlation information. For example, the guidance acquisition unit182 may fetch the message from the correlation information, and maytranslate the message into a language which the authentication targetuses to acquire the guidance.

In addition, for example, the data acquisition unit 110 may fetchattribute data indicating the nationality of the authentication targetfrom the passport, and may translate the message into a languagecorresponding to the nationality that is fetched to acquire the guidanceof a language corresponding to the nationality. As described above, theguidance acquisition unit 182 may acquire guidance corresponding toattribute indicated by the attribute data that is acquired by the dataacquisition unit 110.

Furthermore, the correlation information storage unit 171 may store themessage according to each of a plurality of languages as the guidance incorrelation information, and the guidance acquisition unit 182 may fetcha message in a language corresponding to the nationality that is fetchedby the data acquisition unit 110 from the correlation information.

The face authentication unit 183 performs the face authentication byusing the face image indicated by the face image data that is acquiredby the data acquisition unit 110, and the face image that is captured bythe photographing unit 120. The face authentication unit 183 performsidentity verification of the authentication target through the faceauthentication. That is, the face authentication unit 183 determineswhether or not a person of the face image indicated by the face imagedata that is acquired by the data acquisition unit 110, and a personindicated by the face image that is captured by the photographing unit120 are the same person.

Furthermore, the face authentication unit 183 may be constituted as apart of the face authentication device 100, or a device independent fromthe face authentication device 100.

The output control unit 184 controls the output unit 130 in order forthe output unit 130 to output the guidance acquired by the guidanceacquisition unit 182.

Furthermore, as in the above-described example, in a case where the faceimage indicated by the face image data acquired by the data acquisitionunit 110 is the eyeglass-wearing image, the output control unit 184 maycontrol the output unit 130 in order for the output unit 130 to outputan eyeglass-wearing instruction that recommends the person to bephotographed to wear eyeglasses. When the person to be photographedwears the eyeglasses in response to the eyeglass-wearing instruction, itis possible to reduce the probability of an error occurring in the faceauthentication that is performed by the face authentication unit 183.

In addition, the output control unit 184 allows the display unit 131 todisplay the face image indicated by the face image data acquired by thedata acquisition unit 110. In addition, the output control unit 184 maycontrol the output unit 130 in order for the output unit 130 to output acondition adjusting instruction that recommends the person to bephotographed to adjust conditions, under which the photographing unit120 captures the face image, to conditions shown in the face imagedisplayed by the display unit 131.

For example, in the example of the eyeglass-wearing instruction, theoutput control unit 184 may allow the display unit 131 to display theface image (face image indicated by the face image data acquired by thedata acquisition unit 110) of the passport.

The person to be photographed can confirm instruction contents of thecondition adjusting instruction with reference to the face image bydisplaying the face image of the passport by the display unit 131.

In addition, the output control unit 184 may control the output unit 130in order for the output unit 130 to output difference informationindicating the difference that is detected by the difference detectionunit 181. For example, as described above, the output control unit 184may control the display unit 131 in order for the display unit 131 todisplay guidance of “facial expression is different”.

According to this, the person to be photographed can understand thedifference between the face image of the passport and the face imagecaptured by the photographing unit 120. When the person to bephotographed understands the difference, it is possible to increase theprobability of understanding a handling method for the difference (forexample, a handling method of reducing the difference).

In addition, the output control unit 184 may allow the display unit 131to display a portion different between the face image (face imageindicated by the face image data acquired by the data acquisition unit110) of the passport and the face image captured by the photographingunit 120 as difference information that is shown in any one or both ofthe face image of the passport and the face image captured by thephotographing unit 120.

For example, in a case where the face image of the passport does notwear eyeglasses, and the face image captured by the photographing unit120 wears eyeglasses, the display unit 131 may display the eyeglasses onthe face image of the passport with a broken line, or may performemphasizing display such things as the eyeglasses in the face imagecaptured by the photographing unit 120 are surrounded by a broken linein accordance with the control of the output control unit 184.

According to this, the person to be photographed can more reliablyunderstand a difference between the face image of the passport and theface image captured by the photographing unit 120, and it is possible toincrease the probability that the person to be photographed canunderstand a handling method for the difference (for example, a handlingmethod of reducing the difference).

In addition, the output control unit 184 may allow the display unit 131to display an image including the face image (face image indicated bythe face image data acquired by the data acquisition unit 110) of thepassport, the latest face image captured by the photographing unit 120,and display of guidance acquired by the guidance acquisition unit 182.

The image P21 in FIG. 3 corresponds to an example of the face image ofthe passport. In addition, the image P23 in FIG. 3 corresponds to anexample of the latest face image captured by the photographing unit 120.In addition, display of a message in the region A21 in FIG. 3corresponds to an example of the display of guidance acquired by theguidance acquisition unit 182.

In addition, the output control unit 184 may allow the display unit 131to display an image including a face image which is captured by thephotographing unit 120 and is used in the face authentication by theface authentication unit 183, and in which an error occurs in the faceauthentication, the latest face image that is captured by thephotographing unit 120, and display of the guidance that is acquired bythe guidance acquisition unit 182.

The image P22 in FIG. 3 corresponds to an example of the face imagewhich is captured by the photographing unit 120 and is used in the faceauthentication by the face authentication unit 183, and in which anerror occurs in the face authentication. In addition, the image P23 inFIG. 3 corresponds to an example of the latest face image that iscaptured by the photographing unit 120. In addition, display of amessage in the region A21 in FIG. 3 corresponds to an example of thedisplay of the guidance that is acquired by the guidance acquisitionunit 182.

According to this, the person to be photographed can confirm whether ornot the cause of an error in the face image, which is captured by thephotographing unit 120 and is used in the face authentication by theface authentication unit 183, and in which an error occurs in the faceauthentication, is eliminated in the latest face image that is capturedby the photographing unit 120.

In addition, the output control unit 184 may allow the display unit 131to display an image including the face image (face image indicated bythe face image data acquired by the data acquisition unit 110) of thepassport, the face image which is captured by the photographing unit 120and is used in the face authentication by the face authentication unit183, and in which an error occurs in the face authentication, the latestface image captured by the photographing unit 120, and display of theguidance acquired by the guidance acquisition unit 182.

The image P21 in FIG. 3 corresponds to an example of the face image ofthe passport. In addition, the image P22 in FIG. 3 corresponds to anexample of the face image which is captured by the photographing unit120 and is used in the face authentication by the face authenticationunit 183, and in which an error occurs in the face authentication. Inaddition, the image P23 in FIG. 3 corresponds to an example of thelatest face image captured by the photographing unit 120. In addition,display of a message in the region A21 in FIG. 3 corresponds to anexample of display of the guidance acquired by the guidance acquisitionunit 182.

In addition, the output control unit 184 may allow the display unit 131to display the guidance acquired by the guidance acquisition unit 182with text information. According to this, the person to be photographedcan understand the guidance through reading-out of display of theguidance with the text information.

Display of a message in the region A21 in FIG. 3 corresponds to anexample of display of the guidance acquired by the guidance acquisitionunit 182 with the text information.

In addition, the output control unit 184 may allow the display unit 131to display the guidance for the person to be photographed with an icon.Display of the arrow B21 in FIG. 3 corresponds to an example of the icondisplay of the guidance for the person to be photographed.

In addition, the correlation information storage unit 171 may storecorrelation information in which a difference in a face position betweenthe face image (face image indicated by the face image data acquired bythe data acquisition unit 110) of the passport and the face imagecaptured by the photographing unit 120, and guidance representing adirection in which the person to be photographed moves the face toreduce the difference in the face position are correlated with eachother. In addition, in a case where the difference detection unit 181detects the difference in the face position between the face image ofthe passport and the face image captured by the photographing unit 120,the output control unit 184 may allow the display unit 131 to displayguidance that shows a direction in which the person to be photographedmoves (particularly, the face is moved) with an arrow.

Display of the arrow B21 in FIG. 3 corresponds to an example of displayof the guidance that shows the direction in which the person to bephotographed moves the face with an arrow.

In addition, the output control unit 184 may allow the voice output unit132 to output the guidance at least with a voice.

According to this, even in a case where the person to be photographed isnot looking at the display screen of the display unit 131, or cannot seethe display screen of the display unit 131 due to visual impairment andthe like, the person to be photographed can understand the guidance.

In addition, the output control unit 184 may allow the output unit 130to output an alarm indicating that the output unit 130 outputs theguidance. For example, the output control unit 184 may allow the displayunit 131 to display the guidance, and may allow the voice output unit132 to output a voice message of “please, check guidance”.

According to this, it is possible to reduce the probability that theperson to be photographed erroneously understand the guidance (forexample, the probability that display of the guidance may beoverlooked).

Next, a description will be provided of an operation of the faceauthentication device 100 with reference to FIG. 7 . FIG. 7 is aflowchart showing an example of a procedure of processing that isexecuted by the face authentication device 100. For example, when thepassport is placed on a passport reader device of the data acquisitionunit 110, the face authentication device 100 initiates the processing inthe drawing.

In the processing in FIG. 7 , the data acquisition unit 110 reads outdata from an IC chip that is embedded in the passport (step S101).Particularly, the data acquisition unit 110 reads out image data of theface image (face image of the passport), which is registered in advance,from the IC chip.

In addition, the photographing unit 120 photographs an authenticationtarget (person to be photographed) with a camera to acquire a face image(step S102).

In addition, the face authentication unit 183 performs faceauthentication by using the face image indicated by the image dataacquired by the data acquisition unit 110, and the face image capturedby the photographing unit 120 (step S103).

In addition, the face authentication unit 183 determines whether or nota result of the face authentication can be obtained, or an error hasoccurred in the face authentication (step S104).

In a case where it is determined that the result of the faceauthentication can be obtained (YES in step S104), the faceauthentication device 100 performs processing based on theauthentication result (step S111).

For example, in a case where authentication succeeds (that is, in a casewhere it is determined that the authentication target is the same as aperson described in the passport), the face authentication device 100displays a message indicating that immigration is permitted. Inaddition, the face authentication device 100 allows a door of a gate tobe opened in order for the authentication target to pass through thegate. On the other hand, in a case where the authentication fails (thatis, in a case where it is determined that the authentication target isdifferent from the person described in the passport), the faceauthentication device 100 displays a message that recommends theauthentication target to contact an attendant.

After step S111, the processing in FIG. 7 is terminated.

On the other hand, in a case where it is determined that an error hasoccurred in the face authentication (NO in step S104), the differencedetection unit 181 detects a difference between the face image obtainedfrom the passport in step S101, and the face image captured by thephotographing unit 120 in step S102 (step S121).

In addition, the guidance acquisition unit 182 acquires guidancerepresenting a handling method with respect to the difference obtainedin step S121 (step S122). For example, as described above, thecorrelation information storage unit 171 stores correlation informationin which the difference between the face images, and the guidancerepresenting the handling method with respect to the difference arecorrelated with each other. In addition, the guidance acquisition unit182 fetches the guidance, which is correlated with the differenceobtained in step S121, from the correlation information.

In addition, the output unit 130 outputs the guidance obtained in stepS122 in accordance with a control of the output control unit 184 (stepS123).

After step S123, the processing returns to step S102.

Furthermore, in a case where an error occurs predetermined number oftimes or more, the face authentication device 100 may performpredetermined processing such as displaying of a message that recommendsthe authentication target to contact an attendant.

Specifically, the face authentication unit 183 counts the number oftimes of determination as occurrence of an error in the faceauthentication in step S104, and determines whether or not the number oftimes is equal to or more than predetermined number of times. In a casewhere it is determined that the number of times of determination asoccurrence of an error in the face authentication in step S104 is equalto or more than the predetermined times, for example, the output controlunit 184 allows the display unit 131 to display a message thatrecommends the authentication target contact the attendant.

According to this, the face authentication device 100 can handle with acase where it is difficult to eliminate the error in the faceauthentication.

As described above, the data acquisition unit 110 acquires the faceimage data. In addition, the photographing unit 120 captures the faceimage of the authentication target. In addition, the correlationinformation storage unit 171 stores correlation information, whichrepresents a correlation between the difference between the face imageindicated by the face image data acquired by the data acquisition unit110 and the face image captured by the photographing unit 120, and theguidance for the person to be photographed with respect to thedifference, in advance.

In addition, the difference detection unit 181 detects the differencebetween the face image indicated by the face image data acquired by thedata acquisition unit 110, and the face image captured by thephotographing unit 120. In addition, the guidance acquisition unit 182acquires guidance that is correlated with the difference detected by thedifference detection unit 181 in correlation information. In addition,the output control unit 184 controls the output unit 130 in order forthe output unit 130 to output the guidance acquired by the guidanceacquisition unit 182.

According to this, in a case where the face authentication device 100cannot appropriately perform the face authentication, it is possible toincrease the probability that the authentication target can understandthe handling method.

Hereinafter, the above-described various examples will be collectivelydescribed.

The data acquisition unit 110 acquires the face image data from thepassport.

According to this, it is not necessary to prepare the face image dataexclusively for the processing in the face authentication device 100,and thus a burden on a manager of the face authentication device 100 isreduced in the above-described viewpoint.

In addition, it is not necessary for the storage unit 170 to store theface image data in advance. Due to this viewpoint, storage capacity ofthe storage unit 170 decreases, and thus the manufacturing cost of theface authentication device 100 is reduced.

In addition, the data acquisition unit 110 acquires attribute dataindicating an attribute of the person to be photographed from thepassport. In addition, the guidance acquisition unit 182 acquiresguidance corresponding to the attribute indicated by the attribute dataacquired by the data acquisition unit 110.

According to this, the face authentication device 100 can outputappropriate guidance in correspondence with the attribute of the personto be photographed.

For example, the data acquisition unit 110 acquires the attribute dataindicating the nationality of the person to be photographed from thepassport. In addition, the guidance acquisition unit 182 acquiresguidance of a language corresponding to the nationality indicated by theattribute data.

According to this, the face authentication device 100 can outputguidance in a language that can be understood by the person to bephotographed.

In addition, in a case where the face image indicated by the face imagedata acquired by the data acquisition unit 110 is an eyeglass-wearingimage, the output control unit 184 controls the output unit 130 in orderfor the output unit 130 to output an eyeglass-wearing instruction thatrecommends the person to be photographed to wear eyeglasses.

When the person to be photographed wears eyeglasses in response to theeyeglass-wearing instruction, a difference between the face imageindicated by the face image data acquired by the data acquisition unit110 and the face image captured by the photographing unit 120 isreduced, and thus it is possible to reduce the probability of an erroroccurring in the authentication that is performed by the faceauthentication unit 183.

Furthermore, before the face authentication unit 183 performs the faceauthentication, the output control unit 184 may allow the output unit130 to output the eyeglass-wearing instruction that recommends theperson to be photographed to wear eyeglasses.

Alternatively, in a case where the face authentication unit 183 performsthe face authentication, and an error occurs in the face authentication,the output control unit 184 may allow the output unit 130 to output theeyeglass-wearing instruction that recommends the person to bephotographed to wear eyeglasses. The eyeglass-wearing instruction inthis case corresponds to an example of guidance.

In addition, the output control unit 184 allows the output unit 130 todisplay the face image indicated by the face image data acquired by thedata acquisition unit 110. In addition, the output control unit 184 maycontrol the output unit 130 in order for the output unit 130 to output acondition adjusting instruction that recommends the person to bephotographed to adjust conditions, under which the photographing unit120 captures the face image, to conditions shown in the face imagedisplayed by the display unit 131.

Since the person to be photographed performs handling of adjusting theconditions, under which the photographing unit 120 captures the faceimage, to conditions shown in the face image displayed by the displayunit 131 according to the condition adjusting instruction, it ispossible to reduce the probability of an error occurring in the faceauthentication performed by the face authentication unit 183.

In addition, the output control unit 184 controls the output unit 130 inorder for the output unit 130 to output difference informationindicating the difference that is detected by the difference detectionunit 181.

According to this, the person to be photographed can understand adifference between the face image indicated by the face image dataacquired by the data acquisition unit 110, and the face image capturedby the photographing unit 120 with reference to the differenceinformation. When the person to be photographed understands thedifference, it is possible to increase the probability capable ofunderstanding a handling method for the difference (for example, ahandling method of reducing the difference).

In addition, the output control unit 184 may allow the display unit 131to display a portion different between the face image indicated by theface image data acquired by the data acquisition unit 110 and the faceimage captured by the photographing unit 120 as difference informationthat is shown in any one or both of the face image indicated by the faceimage data acquired by the data acquisition unit 110 and the face imagecaptured by the photographing unit 120. For example, in a case where theface image (face image indicated by the face image data acquired by thedata acquisition unit 110) of the passport does not wear eyeglasses, andthe face image captured by the photographing unit 120 wears eyeglasses,the display unit 131 may display the eyeglass on the face image of thepassport with a broken line, or may perform emphasizing display suchthings as the eyeglasses in the face image captured by the photographingunit 120 are surrounded with a broken line.

According to this, the person to be photographed can more reliablyunderstand a difference between the face image of the passport and theface image captured by the photographing unit 120. Since the person tobe photographed understands the difference, it is possible to increasethe probability that the person to be photographed can understand ahandling method for the difference (for example, a handling method ofreducing the difference).

In addition, in a case where an error occurs in the face authenticationthat is performed by the face authentication unit 183, the differencedetection unit 181 can estimate (detect) a difference that becomes thecause of the face authentication error, the difference being adifference between the face image indicated by the face image dataacquired by the data acquisition unit 110 and the face image captured bythe photographing unit 120.

Since the output control unit 184 controls the output unit 130 in orderfor the output unit 130 to output a handling method corresponding to thedifference, the person to be photographed can understand a handlingmethod corresponding to the cause of the face authentication error (forexample, a handling method of reducing the difference that becomes thecause of the error in the face authentication).

In addition, the output control unit 184 allows the display unit 131 todisplay an image including the face image indicated by the face imagedata acquired by the data acquisition unit 110, the latest face imagecaptured by the photographing unit 120, and display of the guidanceacquired by the guidance acquisition unit 182.

Since the display unit 131 displays the guidance, the person to bephotographed can understand a handling method for the difference betweenthe face image indicated by the face image data acquired by the dataacquisition unit 110 and the face image captured by the photographingunit 120. In addition, when the display unit 131 displays the face imageindicated by the face image data acquired by the data acquisition unit110, and the latest face image captured by the photographing unit 120,the person to be photographed can also understand the difference betweenthe face image indicated by the face image data acquired by the dataacquisition unit 110 and the face image captured by the photographingunit 120, and thus it is possible to increase the probability that theperson to be photographed can understand a handling method for thedifference.

In addition, the output control unit 184 allows the display unit 131 todisplay an image including a face image which is captured by thephotographing unit 120 and is used in the face authentication by theface authentication unit 183, and in which an error occurs in the faceauthentication, the latest face image that is captured by thephotographing unit 120, and display of the guidance that is acquired bythe guidance acquisition unit 182.

According to this, the person to be photographed can confirm whether ornot the cause of an error in the face image, which is captured by thephotographing unit 120 and is used in the face authentication by theface authentication unit 183 and in which an error occurs in the faceauthentication, is eliminated in the latest face image that is capturedby the photographing unit 120.

In addition, the output control unit 184 may allow the output unit todisplay an image including the face image indicated by the face imagedata acquired by the data acquisition unit 110, the face image which iscaptured by the photographing unit 120 and is used in the faceauthentication by the face authentication unit 183, and in which anerror occurs in the face authentication, the latest face image capturedby the photographing unit 120, and display of the guidance acquired bythe guidance acquisition unit 182.

Since the display unit 131 displays the face image indicated by the faceimage data acquired by the data acquisition unit 110, and the face imagewhich is captured by the photographing unit 120 and is used in the faceauthentication by the face authentication unit 183, and in which anerror occurs in the face authentication, the person to be photographedcan confirm a difference between the two face images. Since the personto be photographed confirms the difference, it is expected that theperson to be photographed can accurately understand the guidance that isdisplayed on the display unit 131.

In addition, the output control unit 184 allows the display unit 131 todisplay the guidance acquired by the guidance acquisition unit 182 withtext information.

According to this, the person to be photographed can understand theguidance through reading-out of display of the guidance with the textinformation.

In addition, the output control unit 184 allows the display unit 131 todisplay the guidance for the person to be photographed with an icon.

As described above, the display unit 131 displays the guidance throughdisplay of an icon such as an arrow, various persons to be photographedwho use different languages can understand the guidance.

In addition, the correlation information storage unit 171 storescorrelation information in which a difference in a face position betweenthe face image indicated by the face image data acquired by the dataacquisition unit 110 and the face image captured by the photographingunit 120, and guidance representing a direction in which the person tobe photographed moves the face to reduce the difference in the faceposition are correlated with each other. In addition, in a case wherethe difference detection unit 181 detects the difference in the faceposition between the face image indicated by the face image dataacquired by the data acquisition unit 110, and the face image capturedby the photographing unit, the output control unit 184 allows thedisplay unit 131 to display guidance that shows the direction with anarrow.

According to this, various persons to be photographed who use differentlanguages can understand the movement direction.

In addition, the output control unit 184 allows the voice output unit132 to output the guidance at least with a voice.

According to this, even in a case where the person to be photographed isnot looking at the display screen of the display unit 131, or cannot seethe display screen of the display unit 131 due to visual impairment andthe like, the person to be photographed can understand the guidance.

In addition, the output control unit 184 allows the output unit 130 tooutput an alarm indicating that the output unit 130 outputs theguidance.

According to this, it is possible to reduce the probability that theperson to be photographed erroneously understand the guidance.

Next, a minimum configuration of the invention will be described withreference to FIG. 8 .

FIG. 8 is a schematic block diagram showing a minimum configuration ofthe guidance acquisition device according to the invention. A guidanceacquisition device 10 shown in the drawing includes a data acquisitionunit 11, a photographing unit 12, a correlation information storage unit13, a difference detection unit 14, a guidance acquisition unit 15, andan output control unit 16.

In the configuration, the data acquisition unit 11 acquires face imagedata. In addition, the photographing unit 12 captures a face image. Inaddition, the correlation information storage unit 13 stores correlationinformation that indicates a correlation between a difference betweenthe face image indicated by the face image data acquired by the dataacquisition unit 11 and the face image captured by the photographingunit 12, and guidance for the person to be photographed with respect tothe difference, in advance. In addition, the difference detection unit14 detects a difference between the face image indicated by the faceimage data acquired by the data acquisition unit 11 and the face imagecaptured by the photographing unit 12. In addition, the guidanceacquisition unit 15 acquires the guidance that is correlated with thedifference detected by the difference detection unit 14 in thecorrelation information. In addition, the output control unit 16controls the output unit to output the guidance acquired by the guidanceacquisition unit 15.

According to this, in a case where a device, which performs the faceauthentication by using the face image indicated by the face image dataacquired by the data acquisition unit 11 and the face image captured bythe photographing unit 12, cannot appropriately perform the faceauthentication, it is possible to increase the probability that theauthentication target can understand a handling method.

Furthermore, all functions or partial functions of the control unit 180,the difference detection unit 14, the guidance acquisition unit 15, andthe output control unit 16 in the above-described embodiment can berealized when the CPU reads out and executes a program.

In this case, the program for realizing the functions may be recorded ina computer-readable recording medium, and the program recorded in therecording medium may be read out and executed by a computer system torealize the functions.

Furthermore, it is assumed that the “computer system” stated hereincludes an OS and hardware such as peripherals.

In addition, the “computer-readable recording medium” represents astorage device such as a portable medium including a flexible disk, amagneto-optical disc, a ROM, a CD-ROM, and the like, and a hard diskdrive that is embedded in the computer system.

In addition, the “computer-readable recording medium” may also include aconfiguration in which a program is dynamically retained for a shorttime as in a communication line in a case where the program istransmitted through a network such as the Internet and a communicationchannel such as a telephone line, and a configuration in which theprogram is retained for a constant time as in a volatile memory in thecomputer system which becomes a server or a client in this case.

In addition, the program may be configured to realize parts of theabove-described functions, may be configured to realize theabove-described functions in a combination with a program that isrecorded in the computer system in advance, or may be realized by usinga programmable logic device such as a field programmable gate array(FPGA).

Hereinbefore, the embodiment of the invention has been described indetail with reference to the accompanying drawings. However, a specificconfiguration is not limited to the embodiment, and includes a designand the like in a range not departing from the gist of the invention.

The entirety or parts of the above-described embodiment can be describedas in the following additional statements, but there is no limitationthereto.

(Additional Statement 1)

A guidance acquisition device, including:

-   -   a data acquisition unit that is configured to acquire face image        data;    -   a photographing unit that is configured to capture a face image;    -   a difference detection unit that is configured to detect a        difference between a face image indicated by the face image data        acquired by the data acquisition unit and the face image        captured by the photographing unit or a candidate of the        difference on the basis of at least one of the face image        indicated by the face image data acquired by the data        acquisition unit and the face image captured by the        photographing unit;    -   a guidance acquisition unit that is configured to acquire        guidance on the basis of the difference or the candidate of the        difference which is detected by the difference detection unit;        and    -   an output control unit that is configured to control an output        unit to output the guidance that is acquired by the guidance        acquisition unit.

(Additional Statement 2)

The guidance acquisition device according to Additional Statement 1,further including:

-   -   a correlation information storage unit that is configured to        store correlation information that indicates a correlation        between the difference between the face image indicated by the        face image data acquired by the data acquisition unit and the        face image captured by the photographing unit, and guidance for        a person to be photographed with respect to the difference, in        advance,    -   wherein the difference detection unit is configured to detect        the difference between the face image indicated by the face        image data acquired by the data acquisition unit, and the face        image captured by the photographing unit, and    -   the guidance acquisition unit is configured to acquire guidance        that is correlated with the difference detected by the        difference detection unit in the correlation information.

(Additional Statement 3)

The guidance acquisition device according to Additional Statement 1 or2,

-   -   wherein the data acquisition unit is configured to acquire the        face image data from a passport.

(Additional Statement 4)

The guidance acquisition device according to Additional Statement 3,

-   -   wherein the data acquisition unit is configured to acquire        attribute data indicating an attribute of a person to be        photographed from the passport, and    -   the guidance acquisition unit is configured to acquire guidance        corresponding to the attribute indicated by the attribute data        acquired by the data acquisition unit.

(Additional Statement 5)

The guidance acquisition device according to Additional Statement 4,

-   -   wherein the data acquisition unit is configured to acquire the        attribute data indicating the nationality of the person to be        photographed, and    -   the guidance acquisition unit is configured to acquire guidance        in a language corresponding to the nationality.

(Additional Statement 6)

The guidance acquisition device according to any one of AdditionalStatements 1 to 5,

-   -   wherein in a case where the face image indicated by the face        image data acquired by the data acquisition unit is an        eyeglass-wearing image, the output control unit is configured to        control the output unit to output an eyeglass-wearing        instruction that recommends a person to be photographed to wear        eyeglasses.

(Additional Statement 7)

The guidance acquisition device according to any one of AdditionalStatements 1 to 6,

-   -   wherein the output control unit is configured to control the        output unit in order for the output unit to display the face        image indicated by the face image data acquired by the data        acquisition unit, and control the output unit to output a        condition adjusting instruction that recommends a person to be        photographed to adjust conditions, under which the photographing        unit is configured to capture the face image, to conditions        shown in the face image that is displayed by the output unit.

(Additional Statement 8)

The guidance acquisition device according to any one of AdditionalStatements 1 to 7,

-   -   wherein the output control unit is configured to control the        output unit to output difference information indicating the        difference detected by the difference detection unit.

(Additional Statement 9)

The guidance acquisition device according to Additional Statement 8,

-   -   wherein the output control unit is configured to control the        output unit to display a portion different between the face        image indicated by the face image data acquired by the data        acquisition unit and the face image captured by the        photographing unit as difference information that is shown in at        least any one of the face image indicated by the face image data        acquired by the data acquisition unit and the face image        captured by the photographing unit.

(Additional Statement 10)

The guidance acquisition device according to any one of AdditionalStatements 1 to 9, further including:

-   -   a face authentication unit that is configured to perform face        authentication using the face image indicated by the face image        data acquired by the data acquisition unit, and the face image        captured by the photographing unit,    -   wherein in a case where an error occurs in the face        authentication that is performed by the face authentication        unit, the difference detection unit is configured to detect a        difference that becomes the cause of the face authentication        error, the difference being the difference between the face        image indicated by the face image data acquired by the data        acquisition unit and the face image captured by the        photographing unit.

(Additional Statement 11)

The guidance acquisition device according to any one of AdditionalStatements 1 to 10,

-   -   wherein the photographing unit is configured to perform        capturing of the face image repetitively, and    -   the output control unit is configured to control the output unit        to display an image including the face image indicated by the        face image data acquired by the data acquisition unit, the        latest face image captured by the photographing unit, and        display of the guidance acquired by the guidance acquisition        unit.

(Additional Statement 12)

The guidance acquisition device according to Additional Statement 10,

-   -   the photographing unit is configured to perform capturing of the        face image repetitively, and    -   the output control unit is configured to control the output unit        to display an image including a face image which is captured by        the photographing unit and is used in the face authentication by        the face authentication unit, and in which an error occurs in        the face authentication, the latest face image that is captured        by the photographing unit, and display of the guidance that is        acquired by the guidance acquisition unit.

(Additional Statement 13)

The guidance acquisition device according to Additional Statement 10,

-   -   wherein the photographing unit is configured to perform        capturing of the face image repetitively, and    -   the output control unit is configured to control the output unit        to display an image including the face image indicated by the        face image data acquired by the data acquisition unit, a face        image which is captured by the photographing unit and is used in        the face authentication by the face authentication unit, and in        which an error occurs in the face authentication, the latest        face image that is captured by the photographing unit, and        display of the guidance that is acquired by the guidance        acquisition unit.

(Additional Statement 14)

The guidance acquisition device according to any one of AdditionalStatements 1 to 13,

-   -   wherein the output control unit is configured to control the        output unit to display the guidance acquired by the guidance        acquisition unit with text information.

(Additional Statement 15)

The guidance acquisition device according to any one of AdditionalStatements 11 to 14,

-   -   wherein the output control unit is configured to control the        output unit to display the guidance for a person to be        photographed with an icon.

(Additional Statement 16)

The guidance acquisition device according to Additional Statement 15,

-   -   wherein the correlation information storage unit is configured        to store correlation information in which a difference in a face        position between the face image indicated by the face image data        acquired by the data acquisition unit and the face image        captured by the photographing unit, and guidance representing a        direction in which the person to be photographed moves the face        to reduce the difference in the face position are correlated        with each other, and    -   in a case where the difference detection unit detects the        difference in the face position between the face image indicated        by the face image data acquired by the data acquisition unit and        the face image captured by the photographing unit, the output        control unit controls the output unit to display the guidance        that shows the direction with an arrow.

(Additional Statement 17)

The guidance acquisition device according to any one of AdditionalStatements 1 to 16,

-   -   wherein the output control unit is configured to control the        output unit to output the guidance at least with a voice.

(Additional Statement 18)

The guidance acquisition device according to any one of AdditionalStatements 1 to 17,

-   -   wherein the output control unit is configured to control the        output unit to output an alarm indicating that the output unit        outputs the guidance

(Additional Statement 19)

A guidance acquisition method that is executed by a guidance acquisitiondevice including a correlation information storage unit that isconfigured to store correlation information that indicates a correlationbetween a difference between a face image indicated by face image dataand a face image obtained through photographing, and guidance for aperson to be photographed with respect to the difference, in advance,the method including:

-   -   a data acquisition step of acquiring the face image data;    -   a photographing step of capturing the face image;    -   a difference detection step of detecting the difference between        the face image indicated by the face image data acquired in the        data acquisition step, and the face image captured in the        photographing step;    -   a guidance acquisition step of acquiring guidance that        represents guidance correlated with the difference detected in        the difference detection step in the correlation information;        and    -   an output control step of controlling an output unit to output        the guidance acquired in the guidance acquisition step.

(Additional Statement 20)

A program that causes a computer including a photographing unit, and acorrelation information storage unit that is configured to storecorrelation information that indicates a correlation between adifference between a face image indicated by face image data and a faceimage obtained through photographing, and guidance for a person to bephotographed with respect to the difference in advance to execute:

-   -   a data acquisition step of acquiring the face image data;    -   a photographing step of capturing the face image by the        photographing unit;    -   a difference detection step of detecting the difference between        the face image indicated by the face image data acquired in the        data acquisition step, and the face image captured in the        photographing step;    -   a guidance acquisition step of acquiring guidance correlated        with the difference detected in the difference detection step in        the correlation information; and    -   an output control step of controlling an output unit to output        the guidance acquired in the guidance acquisition step.

INDUSTRIAL APPLICABILITY

According to the invention, in a case where a device that is configuredto perform face authentication cannot appropriately perform the faceauthentication, it is possible to increase the probability that anauthentication target can understand a handling method.

REFERENCE SIGNS LIST

-   -   10 Guidance acquisition device    -   11, 110 Data acquisition unit    -   12, 120 Photographing unit    -   13, 171 Correlation information storage unit    -   14, 181 Difference detection unit    -   15, 182 Guidance acquisition unit    -   16, 184 Output control unit    -   100 Face authentication device    -   130 Output unit    -   131 Display unit    -   132 Voice output unit    -   170 Storage unit    -   180 Control unit    -   183 Face authentication unit

The invention claimed is:
 1. A face authentication system comprising: acamera configured to capture second and third face images of a target; areader configured to read out a first face image from an identificationdocument or an integrated circuit (IC) chip embedded within theidentification document; a display; a memory storing instructions; and aprocessor configured to execute the instructions to: acquire the firstface image of the target from the reader; after acquiring the first faceimage, modify the first face image in which an object worn by the targetis emphasized when the first face image includes the target wearing theobject, and cause the display to display the modified first face image;after causing the display to display the modified first face image,cause the camera to capture the second face image of the target standingin front of the camera; detect a difference between the first face imageand the second face image; cause the display to display, based on thedetected difference, first guidance that recommends the target to takeoff the object worn by the target; perform a face authentication usingthe first face image and the third face image captured by the cameraafter the first guidance is output; and control an opening and closingof a gate according to a result of the face authentication, such thatopening of the gate is permitted when the face authentication issuccessful, and such that opening of the gate is not permitted when theface authentication is unsuccessful.
 2. The face authentication systemaccording to claim 1, wherein the processor is configured to: acquirethe first face image and attribute data of the target from a passport orthe integrated circuit chip embedded within the passport; and cause tothe display to display the first guidance corresponding to an attributeindicated by the attribute data.
 3. The face authentication systemaccording to claim 1, wherein the processor is further configured to:cause to the display to display the first face image and the second faceimage when outputting the first guidance.
 4. The face authenticationsystem according to claim 3, wherein the processor is configured to:cause to the display to display, based on the detected difference, atleast one of the first face image and the second face image, in whichsuch that a portion corresponding to the detected difference isemphasized.
 5. The face authentication system according to claim 1,wherein the processor is further configured to: store the first guidancefor each of a plurality of differences; and acquire the stored firstguidance for the detected difference, and output the acquired firstguidance.
 6. The face authentication system according to claim 1,wherein the processor is configured to: detect the difference in facialexpression between the first face image and the second face image; andcause to the display to display the first guidance that recommends thetarget to make the facial expression expressionless.
 7. The faceauthentication system according to claim 1, wherein the processor isconfigured to: detect, as the difference, whether or not a mask is wornbetween the first face image and the second face image; and cause to thedisplay to display the first guidance that recommends the target to takeoff the mask.
 8. The face authentication system according to claim 1,wherein the processor is configured to: detect a distance between thetarget and the camera based on a gap size between two eyes of the targetin the second face image; and cause to the display to display the firstguidance corresponding to the detected distance.
 9. The faceauthentication system according to claim 1, wherein the processor isconfigured to: cause to the display to display the first guidanceincluding a condition adjusting instruction that recommends the targetto adjust a capturing condition for the camera to a condition shown inthe first face image.
 10. The face authentication system according toclaim 1, wherein the processor is configured to: detect the differencein a face position between the first face image and the second faceimage; and cause to the display to display the first guidance indicatinga movement direction for the target.
 11. The face authentication systemaccording to claim 1, wherein the processor is configured to: performthe face authentication using the first face image and the second faceimage; and control the gate to open when the face authentication usingthe first face image and the second face image is successful, output asecond guidance for the target to contact an attendant when the faceauthentication fails, and cause to the display to display the firstguidance and cause to the camera to capture the third face image when anerror occurs in the face authentication.
 12. The face authenticationsystem according to claim 1, wherein the processor is configured to:cause to the display to display a second guidance for the target tocontact an attendant when an error occurs predetermined number of times.13. The face authentication system according to claim 1, wherein theprocessor is configured to: cause the display to display the second faceimage, which is a horizontally inverted image of a captured image of thetarget standing in front of the camera during capturing, while the firstface image is displayed on the display.